1. Field of Invention
The present invention relates broadly to the field of super resolution. More specifically, it relates to the creation of an output image of higher resolution than an input image using an object model.
2. Description of Related Art
In the field of image processing, it is often desirable to reconstruct a face from a captured image. Preferably, one would desire a high resolution construct of the face, but typically, the captured images come from low resolution sources and/or images where the desired subject's face occupies a small part of the entire image. Basically, a captured image of a desired subject is often a low resolution image, so even if one were to identify and extract the desired subject, the extracted face would be small and/or low resolution. Increasing the size of the image would merely increase the distortion and thus not result in a high resolution rendition.
There is therefore a growing interest in the field of constructing a high resolution images from low resolution images. These types of constructions are typically referred to as super resolution images. This naturally entails adding information not found in the originally captured image. Consequently, one typically needs to know the class of subject one intends to reconstruct in order to have an idea of what type of new information would likely be added. For example, in any given approach, one would need to know the class of object (i.e., a human face, a specific human organ, etc.) whose resolution one intended to increase.
Various approaches have been proposed to achieve this aim, and many of these proposals take a statistical approach. For example, in “A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model” by Liu et al. (State Key Lab of Intelligent Technology and Systems, Dept. of Automation and Visual Computing Group, Microsoft Research China, ISBN 0-7695-1272-0/01, 2001, IEEE), a method for constructing a high resolution face from a low resolution face implements a two step approach utilizing a global linear model relating high-resolution faces to low resolution faces followed by a patch-based markov network. The super-resolution face is created by finding the maximum aposteriori solution of the statistical framework.
Another example is found in “Example-Based Super-Resolution” by Freeman et al., IEEE Computer Graphics and Applications, March/April 2002. Freeman et al. construct low and high resolution dictionaries from sample images, but the dictionaries are not specialized to any specific part of a face.
Another approach is described in “Coupling Face Registration and Super-Resolution” by Jia et al, Department of Computer Science Queen Mary, University of London, London, E1 4NS, UK, 2006. Jia et al propose a multi-resolution tensor model-based approach to super-resolution, and integrate the super-resolution estimation with face registration/alignment.
Still another approach is described in “Limits on Super-Resolution and How to Break Them”, by Baker et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 9, September, 2002, pp. 1167-1183. Baker et al. propose a Bayesian formulation to the problem of making a super-resolution image from a low resolution image of a face. They use multiple features including the original image, 2nd order derivatives and 1st order derivatives.
The above approaches are computationally intensive, which limits their use. What is needed is a more direct approach that does not rely as heavily on statistical modeling to create information to be added to a low resolution image when constructing a higher resolution image (i.e., super-resolution image) from the low resolution image.